feat: user cost by wjiayis · Pull Request #126 · PaperDebugger/paperdebugger

wjiayis · 2026-02-19T11:58:57Z

Junyi-99 · 2026-02-22T09:12:14Z

Hi @wjiayis , thanks for another great contribution!

regarding your question: yes, we do need a per-user usage cap.

thanks again!

wjiayis · 2026-02-22T09:14:41Z

@Junyi-99 No problem! Yea cool, feel free let me know the token limit you decide to have! I feel that having some sort of progress bar would make more sense than the absolute token number.

Junyi-99 · 2026-02-22T09:27:19Z

@wjiayis yeah, a progress bar is much more intuitive than raw numbers.

Since models vary in pricing, let's implement a USD-based cap: Per session: $1, Per week: $2, Per month: $3

Since reasoning models make it hard to calculate exact overhead, let’s also add a small disclaimer in the UI mentioning that the usage is an estimate. We'll run this for a bit and adjust based on the results.

Also, we should support different limits for different users (e.g., tiered usage caps).

What are your thoughts on this tiered pricing approach? Any other suggestions for the plan?

Thanks!

Copilot

Pull request overview

This pull request implements user-level token usage tracking to address Issue #58, enabling the system to distinguish between heavy and light users and track LLM token consumption per user. The implementation adds session-based tracking (5-hour windows) and weekly aggregation, with both backend services and a frontend UI to display usage statistics.

Changes:

Added usage tracking service that records token consumption from OpenAI API responses and stores them in MongoDB with session-based aggregation
Created new gRPC/REST API endpoints to retrieve current session and weekly usage statistics
Implemented frontend Usage tab to display token consumption metrics with auto-refresh capability

Reviewed changes

Copilot reviewed 28 out of 29 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
proto/usage/v1/usage.proto	Defines protobuf messages and service for usage tracking API
pkg/gen/api/usage/v1/*.go	Generated Go code for usage service (gRPC server, gateway, protobuf messages)
webapp/_webapp/src/pkg/gen/apiclient/usage/v1/usage_pb.ts	Generated TypeScript protobuf types for frontend
internal/models/usage.go	MongoDB model for LLM session tracking
internal/services/usage.go	Core usage tracking service with session management and weekly aggregation
internal/api/usage/*.go	API handlers for session and weekly usage endpoints
internal/libs/db/db.go	Database index creation for TTL and efficient session lookups
internal/services/toolkit/client/completion_v2.go	Integration to capture usage data from OpenAI streaming responses
internal/services/toolkit/client/utils_v2.go	Enables usage reporting in stream options
internal/services/toolkit/client/client_v2.go	Dependency injection for usage service
internal/services/toolkit/client/get_conversation_title_v2.go	Updated to pass userID for usage tracking
internal/services/toolkit/client/get_citation_keys.go	Updated to pass userID for usage tracking
internal/wire.go, internal/wire_gen.go	Dependency injection wiring for usage service
internal/api/server.go, internal/api/grpc.go	Registration of usage service endpoints
webapp/_webapp/src/views/usage/index.tsx	Frontend UI component displaying usage statistics
webapp/_webapp/src/query/*.ts	React Query hooks and API client functions for usage endpoints
webapp/_webapp/src/paperdebugger.tsx	Added Usage tab to main navigation
pkg/gen/api/chat/v2/chat.pb.go	Generated code formatting change (import order)
internal/api/chat/create_conversation_message_stream_v2.go	Updated to pass userID for usage tracking
internal/services/toolkit/client/get_citation_keys_test.go	Test setup updated with usage service dependency
go.sum	Updated Go module dependencies

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/libs/db/db.go

internal/services/toolkit/client/completion_v2.go

internal/services/usage.go

wjiayis · 2026-02-23T13:58:23Z

@Junyi-99 Sure, a USD-based cap makes a lot of sense!

Also, we should support different limits for different users (e.g., tiered usage caps).

Are employing a freemium model and charge heavy-usage users? I think it's a great way to monetize PaperDebugger, balancing ease of onboarding for new users, API costs, and returns for sustained development/maintenance.

Just off the top of my head, I suggest we could roll this out in 2 phases

We deploy the token counter on the backend but not display it on the frontend, to observe usage distribution
Based on the usage distribution, we launch 3 tiers of usage: Free, Lite and Pro.
a. Free tier could have the limits of Per session: $1, Per week: $2, Per month: $3
b. Lite tier could be a reasonable limit that encompass almost all users
c. Pro tier could be a really high limit for a few users
We could also make XtraMCP available exclusively to paid users to increase incentives for subscribing

Alternatively if we decide not to monetize PaperDebugger, I could block all requests past the free tier usage.

4ndrelim

Thanks for the good work Jia yi! I haven't yet finished looking through the files, and some parts im not too sure yet, but I will approve first for integration testing on staging and do a deeper review in the meantime before the final PR to main.

internal/models/usage.go

internal/services/toolkit/client/completion_v2.go

wjiayis · 2026-03-03T12:27:39Z

@Junyi-99 @4ndrelim Shall we discuss how to move forward with this feature over a call? I think some key points are

How do we prevent / disincentivize users from bypassing our limits easily
Are we planning to use this as a form of monetization or purely to prevent abuse

Junyi-99 · 2026-03-04T08:17:24Z

@Junyi-99 Sure, a USD-based cap makes a lot of sense!

Also, we should support different limits for different users (e.g., tiered usage caps).

Are employing a freemium model and charge heavy-usage users? I think it's a great way to monetize PaperDebugger, balancing ease of onboarding for new users, API costs, and returns for sustained development/maintenance.

Just off the top of my head, I suggest we could roll this out in 2 phases

We deploy the token counter on the backend but not display it on the frontend, to observe usage distribution

Based on the usage distribution, we launch 3 tiers of usage: Free, Lite and Pro.
a. Free tier could have the limits of Per session: $1, Per week: $2, Per month: $3
b. Lite tier could be a reasonable limit that encompass almost all users
c. Pro tier could be a really high limit for a few users

We could also make XtraMCP available exclusively to paid users to increase incentives for subscribing

Alternatively if we decide not to monetize PaperDebugger, I could block all requests past the free tier usage.

@wjiayis If we’re treating PaperDebugger as an independent project, I’m aligned with moving toward a freemium model and charging heavy users. Your plan is quite solid.

The 2-phase rollout (observe first, then design tiers based on real data) is very reasonable. I’m good to move forward with Phase 1 and collect usage data first. Then we can refine Free / Lite / Pro based on actual distribution.

wjiayis self-assigned this Feb 19, 2026

wjiayis added the enhancement New feature or request label Feb 19, 2026

wjiayis changed the base branch from main to staging February 19, 2026 12:00

wjiayis force-pushed the feat/user-cost branch 3 times, most recently from b9bda9c to 4206592 Compare February 21, 2026 06:43

feat: LLM usage tracking, backend

d93462e

wjiayis force-pushed the feat/user-cost branch from 4206592 to d93462e Compare February 21, 2026 06:51

wjiayis added 4 commits February 21, 2026 15:01

feat: LLM usage tracking, frontend

3a73435

feat: simplify frontend display

c882e6e

feat: refresh handling

baeeaa7

chore: update comments

27092ae

wjiayis force-pushed the feat/user-cost branch from ff74314 to 27092ae Compare February 21, 2026 08:24

wjiayis marked this pull request as ready for review February 21, 2026 08:26

wjiayis requested review from 4ndrelim and Junyi-99 and removed request for Junyi-99 February 21, 2026 08:26

Junyi-99 previously approved these changes Feb 22, 2026

View reviewed changes

4ndrelim requested a review from Copilot February 23, 2026 13:32

Copilot started reviewing on behalf of 4ndrelim February 23, 2026 13:33 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

chore: clarify comments

3c6a7f2

wjiayis dismissed Junyi-99’s stale review via 3c6a7f2 February 23, 2026 14:02

4ndrelim previously approved these changes Feb 23, 2026

View reviewed changes

internal/models/usage.go Show resolved Hide resolved

internal/services/toolkit/client/completion_v2.go Outdated Show resolved Hide resolved

internal/services/toolkit/client/completion_v2.go Show resolved Hide resolved

feat: add usage_test.go

4660a29

wjiayis dismissed 4ndrelim’s stale review via 4660a29 February 25, 2026 13:46

wjiayis added 2 commits February 25, 2026 22:05

feat: improve llm_sessions compound index

92a99ad

chore: improve race condition error handling

736871b

wjiayis added 4 commits March 4, 2026 20:19

feat: break down to per-model usage

ea8782f

feat: display USD token prices

18402b2

feat: improve display of usage costs

219396f

fix: only track usage for non-BYOK

dc78d5b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: user cost#126

feat: user cost#126
wjiayis wants to merge 13 commits intostagingfrom
feat/user-cost

wjiayis commented Feb 19, 2026 •

edited

Loading

Uh oh!

Junyi-99 commented Feb 22, 2026

Uh oh!

wjiayis commented Feb 22, 2026

Uh oh!

Junyi-99 commented Feb 22, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wjiayis commented Feb 23, 2026 •

edited

Loading

Uh oh!

4ndrelim left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wjiayis commented Mar 3, 2026

Uh oh!

Junyi-99 commented Mar 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

wjiayis commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Junyi-99 commented Feb 22, 2026

Uh oh!

wjiayis commented Feb 22, 2026

Uh oh!

Junyi-99 commented Feb 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wjiayis commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

4ndrelim left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wjiayis commented Mar 3, 2026

Uh oh!

Junyi-99 commented Mar 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

wjiayis commented Feb 19, 2026 •

edited

Loading

Junyi-99 commented Feb 22, 2026 •

edited

Loading

wjiayis commented Feb 23, 2026 •

edited

Loading